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by the speaker in the course of the speech. A speech recog- 
nition system operating on a speaker device (i.e., a comput- 
ing system a with a microphone connected to it) recognizes 
during the speech (i.e., word spotting) the pronunciation by 
the speaker of anyone of said hyperlinked terms, and records 
the time at which each recognized hyperlinked term has been 
pronounced The system is also based on the synchroniza- 
tion of the speaker device with several auditors devices (e.g., 
workstations, portable computers, personal digital assistants 
- PDAs , smart phones, or any other type of handheld comput- 
ing devices) according to a same universal time, so that the 
flow of information transmitted by the speaker and received 
by the auditors is always synchronized, independently of the 
relative positions of the speaker and auditors. Each time the 
auditor perceives an interesting topic during the speech, he 
immediately selects the topic simply by pressing a reserved 
key on the auditor's device. Universal times at which topics 
are selected by the auditor are stored in the auditor device. 
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SYSTEM AND METHOD FOR ENHANCING LIVE SPEECH WITH 
INFORMATION ACCESSED FROM THE WORLD WIDE WEB 

. Field of the invention 

The present invention relates generally to tecliniques for integrating in a iive 
5 event, means for accessing complementary information, and more, particularly to 
systems and methods for enhancing a live event, such as a live speech or a live 
broadcast program, with Information, in particular with information that can be 
accessed from the World Wide Web. 

Background of tfie invention 

10 Need for Information 

In our society, people have a growing need for information and services. Most 
spectators attending to a live event (e.g., as auditors of a conference given by a 
speaker), or auditors attending to a live radio or television broadcast program, want 
to have access to complementary infomiation. This complementary information may 
15 consist in the biography of a player of a football match, the historical background on 
events cited on a news program, or athletic records during the transmission of an 
Olympic competition. 

In fact, today people are looking for more information about what they are 
hearing or listening locally, as spectators of live events, or remotely, as auditors of 
20 live broadcast programs : 

• Consumers want to have access to special sendees associated with 
advertised products. 

• Media providers expect new sources of profit by extending the quantity and 
quality of services and information provided to consumers and more 

25 particularly to auditors of live television or radio programs. 

• Advertisers are looking for new and more effective fomis of advertisement. 



) 



wo 2004/029831 




wo 2004/029831 0 ^ ^ PCT/EP2003/009310 

On-line Services on the Web 

Independently of the massive development of radio and television, on-line 
services such as those provided on the World Wide Web (i.e., the Web), have 
rapidly emerged in our society and are now widely available. Such on-line sendees 

5 based on the Internet technology, provide access to a huge amount of information 
on an interactive basis. The Internet Is a global network of computer. The Internet 
connects computers based on a variety of different operating systems or languages 
using a language referred to as TCP/IP (Transmission Control Protocol/lntemet 
Protocol"). With the increasing size and complexity of the Intemet, tools have been 

10 developed to help users to find infonnation they need on the network. These tools 
are often called "navigators'* or "navigation systems". The World Wide Web ("WWW" 
or "the Web") is a recent superior navigation system. The Web Is : 

• an Internet-based navigation system, 

• an information distribution and management system for the Intemet, and 
15 • a dynamic fomriat for communicating on the Web. 

Intemet and the Web is transforming our society by offering to millions of 
users, the opportunity to access and exchange infonnation and to communicate 
between each other. By integrating images, text, audio and video, a user on the 

20 Web using a graphical user interface can today transparently communicate with 
different computers on the system, different system applications, and different 
information formats for files and documents including, for example, text, sound and 
graphics. Currently, on-line systems on the Web offer a variety of dHferent 
sendees to users, for instance, private message sewices. electronic commerce, 

25 news, real-time games, access to electronic databases, electronic newsletters, 
business-to-business transactions, or job placement sen/ices. 

But, even if such on-line sen/ices are now available, the searching and finding 
of the relevant information on the Web remains an atxluous task, sometimes taking 
hours, even for experienced users. Obviously, since the Web is essentially an open, 
30 multi-point to multi-point networic. each user can select and retrieve different 
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information from many different servers. In fact, today, most on-line interactions with 
the Web occur merely through textual inputs for instance by entering URLs (Uniform 
Resource Locator) names, by entering key words on search tools, or by activating 
textual hyperlinks on HTML (Hypertext Mart<up Language) documents. Even If in a 

5 near future, the development of audiovisual interfaces (e.g., human speech 
interfaces. Web-phone integration) will render textual inputs less and less dominant 
in on-line environments, there is a good chance that the Web remains user 
unfriendly due to its massiveness, its lack of organization, and its randomness. 
Simply stated, in the Web. there is no order or direction. Infomfiation remains most of 

10 the time hard to find and even worse, in a foreseeable future, it will remain a difficult 
task to find the required infomiation into the desired context. 

On-line Services from Live Speech 

Unlike the multi-point to multi-point Web network, a live speech to an 
audience (the audience being in the same location as the speaker or the audience 

15 being remotely located, i.e., accessed through a radio or television broadcast 
station) is primarily a communication from a single emitter to multiple receivers. 
Every auditor receives the same content, locally from the speaker, or remotely 
through the broadcasting station. 

Thus, to provide on-line sen/ices similar to those that can be accessed on the 

20 Web, a first problem in a live speech is that the information flows continuously in the 
same direction, from a single source to multiple receivers, from a provider to multiple 
auditors. The communication flow is limited to one direction without any. exchange of 
information with the auditors. People cannot directly interact with the oral information 
that is received, to access additional information or sen/ices. 

25 Moreover, when people hear a live speech, a problem for the auditors is to 

select topics of interest and then to identify the network addresses (i.e., URLs) to 
access (e.g., from the Web) the multimedia information or services related with the 
selected topics. Until today, this problem has been partially solved. 



) 



wo 2004/029831 




wo 2004/029831 




PCT/EP2003/009310 



To provide web-like capabilities to the oral or radio information, a solution is to 
embed infonnation (e.g., URLs) into the transmitted broadcast audio signals or on 
separate channels (simultcast). Examples of such systems are described in the 
following patents : US patent 6,125,172 entitled "Apparatus and method for initiating 

5 a transaction having acoustic data receiver that filters human voice", US patent 
6,098,106 entitled "Method for controlling a computer with an audio signal", US 
patent 5,841,978 entitled "Network linking method using steganographically 
embedded data objects", US patent 5,832,223 entitled "System, method and device 
for automatic capture of Internet access infonnation in a broadcast signal for use by 

10 an Internet access device", US patent 5,761,606, entitled "Media online services 
access via address embedded in video or audio program", US patent 5,189,630 
entitled "Method for encoding and broadcasting infonnation about live events using 
computer pattern matching techniques", US patent 5,119,507 entitled "Receiver 
apparatus and methods for Identifying broadcast audio program selections in a radio 

15 broadcast system" or US patent 6,061,719 entitled "Synchronized presentation of 
television programming and web content". 

The systems and methods described in these patents require the 
transmission of a complementary infonnation (e.g., URLs) encoded, embedded or 
modulated on the same audio or video signal or transmitted on a separate channel, 
20 concurrently with the transmission of the main program. Radio or television stations 
must comprise means for encoding, modulating and transmitting along with the 
audio signal this complementary infomnation. The radio auditors or television viewers 
must be equipped with special receivers and decoder circuits for recovering this 
information. 

25 Independently of the herein above discussed arrangements, systems have 

been developed to enable auditors to "pre^se/ecT topics of Interest (i.e., keywords or 
sentences) and to associate these topics with pre-specified network addresses (i.e., 
URLs). These pre-specified network addresses are used to access multimedia 
information or services related with the pre-selected topics. In general temis, all 

30 these system are based on speech recognition techniques. These techniques are 
used to identify keywords (i.e., selected words or sentences) for perfonning specific 
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actions in response to the recognition of specific sounds. Examples of tliese 
systems can be found in tlie following patents : 

US patent 5,946,050 entitled "Keyword listening device" discloses a method 
and a system for monitoring the audio portion of a broadcast signal by means of a 

5 keyword listening device, where a relatively limited set of keywords are stored. The 
keyword listening device monitors the broadcast signal for any of these keywords. 
Upon recognition of any one or more of the keywords, the broadcast audio signal is 
recorded for a period of time and then fully analyzed. After analysis, and in 
dependence uplon the recorded and analyzed broadcast audio signal, a number of 

10 different functions, such as connection to an external network at a.specified address, 
or control of a video cassette recorder, may be performed. 

US patent 6,011,854 entitled "Automatic recognition of audio information in a 
broadcast program" discloses an audio processing system to search for information 
reports or updates (such as traffic, weather, time, sports, news, and the like) 

15 broadcast over one or several radio stations. The search is based on at least one 
keyword (such as "traffic", "weather", "time", "sports", "news" depending on the 
desired report) pre selected by the user, and entered into the audio processing 
system. While speech recognition software used by the audio processing system 
scans the radio station for the requested infomiation report, the user may listen to 

20 other audio sources (a CD, a tape, another radio station broadcast, etc.) without 
being required to monitor (that is, listen to) the information content from those audio 
sources. Once the requested information report is detected based on the entered 
keyword used in the radio broadcast, the audio processing system switches Its audio 
output to the radio station transmitting the desired broadcast, so that the user can 

25 timely and conveniently listen to the traffic, weather, time, sports, and/or news 
reports or updates. 

US patent 6,332;i20 entitled "Broadcast speech recognition system for 
keyword monitoring" discloses a system where broadcast audio is automatically 
monitored for infomiation of Interest. This systeip comprises a computer processor 
30 with a memory for storing a vocabulary of keywords of interest, an audio receiver for 
receiving an audio broadcast and a speech recognition system, associated with the 



5 



WO 2004/029831 PCT/EP2003/009310 

6 

audio receiver and the computer processor, for detecting when one of the keywords 
appears in a received audio segment. A report generator associated with the 
computer processor and responsive to the detection of a keyword, generates a 
report with details related to the detected keyword and its context. 

Even if the systems previously mentioned don't require the transmission of 
complementary infomiation embedded with the audio signal {or on a secondary 
signal concurrently with the retransmission of the main program), the auditors must 
be equipped with receivers with speech recognition capabilities to detect the 
occurrence of hyperlinked temis in the data stream. 



10 



In the field of speech processing, the ability to identify occurrences of words 
or sentences in a stream of voice data is commonly called "word spotting". The goal 
of audio wprdspotting is to identify the boundaries of a search temi within a digitized 
continuous speech stream without prior manual transcription. Searching and 
indexing a live speech that may be pronounced by any speaker, is particulariy 
15 problematic. This is due in large part, to the limited capabilities of the existing 
automatic speech recognition technology. It is important to note that in the above 
discussed systems, the word spotting task is. done on the auditor side, in a speaker 
independent manner, with unrestricted vocabulary, and employ speech models 
trained using voice data other than the data to recognize. 

20 In fact, a fundamental problem with all systems, is the unreliable behavior of 

the state-of-the-art speech recognition technology for performing "word spotting' 
(i.e., identification of pre-specifled keywords or tenns) on a continuous manner, 
independently to the speaker, based on unknown or generic speaking styles, 
vocabularies, noise levels and language models. 

25 As shown in the foregoing discussion, even if during these last years, 
interactive systems have been developed for increasing and improving the level of 
interaction with users and for providing more infomiation and more learning or 
entertainment opportunities (e.g.. interactive television. WebTV), important sources 
of Information, such as those that can be found in the Web, still remain inaccessible 
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for auditors of a live speech (e.g., a live conference or a live interview received from 
a radio or television broadcast). 

Therefore, today there is a need to provide a convenient, universal, and easy 
mechanism for enabling people attending a live speech (e.g., or receiving a live 
5 broadcast program) to select and access complementary infomiation. 

There is also a need for speakers and producers of live broadcast programs 
to create hyperlinks from selected terms (generally selected spoken utterances, 
words or sentences) intended to be pronounced during a speech (e.g., on the 
course of a conference or during a live radio or television program), to relevant data 
10 on the Web without embedding these hyperlinks in conventional one-way broadcast 
signals, and more generally without physically transmitting these hyperlinks and 
without modifying conventional transmitters or receivers. 

Objects of the invention 

It is a broad object of the invention to enhance audio information such as a 
15 live speech or a live radio or television broadcast program, with complementary 
information or services related to this audio infomnation. 

It is another object of the invention to create hyperlinks between selected 
terms or words intended to be pronounced by a speaker in the course of a speech 
and complementary infomnation related to these selected temris. 

20 It is another object of the invention to identify hyperlinked terms as they are 

. pronounced by a speaker in the course of a speech, and to activate the hyperlinks 
that are associated with the identified tenms. 

It is another object of the invention to enable an auditor to select during the 
course of a live speech, one or several terms related to topics drawing his attention 
25 and, immediately after said speech or at a later time, to access information related to 
the topics previously selected. 
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It Is a further object of the invention to enable an auditor of a live speech to 
access information related to topics that have drawn his attention during the speech 
with the minimum effort and involvement from his part while minimizing the 
complexity of the required equipment. 

5 Summary of the Invention 

The present invention is directed to a system, method and computer program 
as defined in independent claims for enabling an auditor of a live speech to access 
immediately after or at a later time, complementary information related with terms 
pronounced during the speech. 

10 The system associates hyperlinks (i.e., URLs) with selected terms or words 

likely to be pronounce by the speaker in the course of the speech. A speech 
recognition system operating on a speaker device (i.e., a computing system a with a 
microphone connected to it) recognizes during the speech (i.e., word spotting) the 
pronunciation by the speaker of anyone of said hyperlinked terms, and records the 

15 time at which each recognized hyperiinked term has been pronounced. 

The system is also based on the synchronization of the speaker device with 
several auditors devices (e.g., workstations, portable computers, personal digital 
assistants - PDAs, smart phones, or any other type of hand held computing devices) 
according to a same universal time, so that the flow of infomnation transmitted by the 
20 speaker and received by the auditors is always synchronized, independently of the 
relative positions of the speaker and auditors. Each time the auditor perceives an 
interesting topic during the speech, he immediately selects the topic simply by 
pressing a reserved key on the auditor's device. Universal times at which topics are 
selected by the auditor are stored in the auditor device. 

25 In a preferred embodiment of the invention, the synchronization between 

speaker and auditor's devices is done referring to an universal time such as the 
Global Positioning System Time (GPS-time), the Global Orbiting Navigational 
Satellite System (GLONASS) time or another suitable universal time based on a 
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satellite system. A GPS or GLONASS receiver is integrated or connected to the 
speaker device. A GPS or GLONASS receiver Is Integrated or connected to each 
auditors device. Each auditor device is independent and separate from the radio or 
television sets used by the auditor to receive the broadcast speech. 

5 More particularly, the present invention discloses a system, method and 

computer program for generating from a speaker device, a Speech Hyperlink-Time 
table during a speech, said Speech Hyperlink-Time table being accessible by one or 
a plurality of auditors. The method comprises the steps of: 

• locating and accessing a Speech Hyperlink table comprising : 

10 • means for identifying one or a plurality of predefined hyperlinked terms 
intended to be pronounced by a speaker; 
• means for locating and accessing information related to each of said one or 
plurality of predefined hyperlinked terms; 

• retrieving means for locating and accessing a Speech HyperlinkrTime table; 

15 

during the speech : 

• recognizing, when they are pronounced by the speaker, the hyperlinked terms 
predefined in the Speech Hyperlink table by means of a speech recognition 
system connected to the speaker device; 

20 for each recognized hyperlinked temn : 

• determining an universal-time corresponding to the pronunciation by the speaker 
of the recognized hyperlinked term; 

• creating a new record on the Speech Hyperlink-Time table; 

• copying into said new record : 

25 • the universal-time corresponding pronunciation by the speaker of the 
recognized hyperlinked term; 
• means, retrieved from the Speech Hyperlink table, for identifying the 
recognized hyperlinked term; 
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means, retrieved from the Speech Hyperlink table, for locating and accessing 
infomiatlon related to the recognized hyperlinked term. 



The present invention also discloses a system, method and computer 
program for selecting from an auditor device, one or a plurality of hyperlinked terms 
5 pronounced by a speaker during the course of a speech, and for accessing, at a 
later time, information related to each of said one or plurality of selected hyperlinked 
temris. The method comprises the steps of: 

• each time a selection command is received for selecting a temi currently 
pronounced by a speaker during the course of a speech : 

10 • detemiining the current universal-time; 

• recording the current universal-time in a Selections Hyperiink-Time table; 

• accfessing a Speech Hyperiink-Time table, said Speech Hyperiink-Time table, 
accessible by one or a plurality of auditor devices, comprising for each one of a 
plurality of predefined hyperiinked terms pronounced by the speaker : 

15 • an universal-time interval corresponding to the pronunciation of the 
predefined hyperiinked tenn; 

• means for identifying the predefined hyperiinked term; 

• means for locating and accessing infonnation related to the predefined 
hyperiinked temn; 

20 for each universal-time recorded in the Selections Hyperiink-Time table: 

• identifying in the Speech Hyperiink-Time table, the selected hyperiinked term 
pronounced at the recorded universal-time; 

• retrieving from the Speech Hyperiink-Time table, said means for identifying and 
for locating and accessing the selected hyperiinked temi; 

25 • storing the retrieved means for identifying the selected hyperiinked term and for 
locating and accessing infonnation related to the selected hyperiinked term into 
the Selections Hyperiink-Time table. 



Further embodiments of the invention are provided in the appended 
dependent claims. 
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The foregoing, together with other objects, features, and advantages of this 
invention can be better appreciated with reference to the following specification, 
claims and drawings. 

5 Brief description of the drawings 

The novel and inventive features believed characteristics of the invention are 
set forth In the appended claims. The invention itself, however, as well as a 
preferred mode of use, further objects and advantages thereof, will best be 
understood by reference to the following detailed description of an illustrative 
10 detailed embodiment when read In conjunction with the accompanying drawings, 
wherein: 

• Figure 1 shows how an auditor of a live speech such as a conference perceives 
topics of interest. 

• Figure 2 shows how an auditor perceives topics of interest when listening to a 
15 live radio or television program. 
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• Figure 3 illustrates how, according to the present invention, the speaker 
workstation and auditors devices are synchronized according to a same universal 
time, and how a speech recognition system operating on the speaker device, 
recognizes hyperlinked terms pronounced by the speaker. 

20 . 

• Figure 4 shows how, according to the present invention, the speaker prepares 
in advance the text of the speech. 

• Figure 5 shows how, according to the present invention, the speaker selects in 
the text of the speech, words or terms for creating hyperlinks (i.e., associate 

15 URLs) to multimedia information or services. 
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Figure 6 shows, how according to the present Invention, the speaker creates a 
Speech Hyperlink Table that associates to each hyperlinked term, an address to 
access multimedia Information or sen/tees. 

Figure 7 shows how, according to the present invention, the speaker the speech 
recognition system trains before the speech. 

Figure 8 shows how, according to the present invention, during the speech, the 
speech recognition system operating on the speaker workstation recognizes (i.e., 
word spots) the hyperlinked temis as they are pronounced by the speaker. 

Figure 9 illustrates how, according to the present Invention, when a hyperlinked 
temi is recognized (word spotted) by the speech recognition system, the 
recognized term, the associated address and the universal time of the 
recognition are stored as in a Speech Hyperlink-Time Table located on a Speech 
Server connected to a communications network (e.g., to the Internet). 

Figure 10 shows an example of a Speech Hyperlink-Time Table stored on a 
Speech Server according to the present invention. 

Figure 11 shows how, according to the present Invention, an auditor perceiving 
a term corresponding to an interesting topic during the course of the speech, 
selects this term simply by pressing a reserved key on a portable computer. 

Figure 12 illustrates how, according to the present invention, the universal times 
at which interesting terms are selected by the auditor during the speech, are 
stored in a Selections Hypertink-Time Table located in the auditor device 
wori<otatio ft. 

Figure 13 shows how, according to the present invention, by connecting the 
auditor device to the communications network, the auditor updates the 
Selections Hyperiink-Time Table located in his wori<station using the information 
comprised in the Speech Hyperiink-Time Table stored on the Speech Sender. 
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Figure 14 shows how. according to the present invention, hyperlinked terms are 
identified and copied, jointly with the associated Uniform resource Locators 
(URLs), from the Speech Hyperlink-Time Table stored on the Speech Server to 
the Selections Hyperlink-Time Table on the auditor device. 

Figure 15 shows how, according to the present invention, an auditor selects from 
the updated Selections Hyperlink-Time Table an hyperlinked term (corresponding 
to an interesting topic selected by the auditor during the speech), and activates 
the associated hyperlink. 

Figure 16 shows how, according to the present invention, multimedia infomiation 
or services hyperlinked to a selected term is accessed through the 
communications network and retrieved on the auditor device. 

Figure 17 shows the steps of creating a Speech Hyperlink Table and of tralnir)g 
a speech recognition system to identify hyperlinked terms, according to the 
present invention. 

Figure 18 shows the steps of creating a Speech Hyperlink-Time Table on a 
Speech Server and of recognizing hyperlinked terms pronounced during a 
speech, according to the present invention. 

Figure 19 shows the steps of creating a Selections Hyperlink-Time Table on the 
auditor device and for selecting topics of interest during the course of a speech, 
according to the present invention. 

Figure 20 shows the steps of retrieving Unifomn Resource Locators (URLs) and 
for having access to the, information or services related to selected hyperiinked 
terms. . 

Preferred embodiment of the invention 
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As shown In Figures 1 and 2, the present invention discloses a system and a 
method for enabling a spectator (100) of a live event (e.g., attending to a 
conference) (102) (202), or a radio auditor or television viewer (200) attending a live 
broadcast program, to select one or several topics (101), (201) drawing his attention 
5 or interest, and to Immediately or at a later time, to easily have access to multimedia 
infomiation related with the selected topics (103), (203). 

Figure 1 shows a typical situation according to the present invention. An 
auditor (100) attending a live speech (e.g., a conference about the subject "Wine & 
Healtli'), has his attention drawn by a topic (101) (e.g., the term "Resveratrol") for 
10 which he would iil<e to have further information. Under these circumstances, there is 
a need to provide a simple mechanism to enable an auditor (100) to select topics 
during the course of a speech (e.g., the terms Tannis", "Phenols", "Resveratrol"), 
and to access immediately or at a later time, infomnation related to the selected 
topics on a sen/er connected, for instance, to the Internet. 

15 Figure 2 illustrates another typical situation. A .television viewer (200) attends 

a live broadcast speech (e.g., to a live television program about the subject 'Wine & 
Health"). 

As shown in Figure 3, hyperlinks (i.e., URLs) (303) are associated with temis 
or words (304) likely to be cited by the speaker (301) in the course of his speech. 

20 These hyperlinked ternis or words are related to relevant topics or Items (302). A 
speech recognition system (305) operating on a speaker workstation (306) (i.e., a 
computing system a with a microphone (307) connected to it) detects during the 
speech, the pronunciation by the speaker of anyone of said hyperlinked terms (308), 
and records the time (309) at which the detected hyperlinked temi has been 

25 pronounced. The system is also based on the synchronization (312) of the speaker 
device (306) with several auditors devices (311) (e.g., workstations, portable 
computers, personal digital assistants - PDAs , smart phones, or any other type of 
hand held computing devices) according to a same universal time (309) (310), so 
that the flow of infomiation transmitted by the speaker (301) and received by the 
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auditors (300) is always synchronized, independently of the relative positions of 
speal<er (301) and auditors (300). Each time an auditor (300) perceives an 
interesting topic (313) during the speech, he immediately selects this topic simply by 
pressing a resen/ed key (314) on the auditor's device (31 1). Univereal times (310) at 
5 which topics are selected (314) by the auditor (300) are stored on the auditor device 
(311). 

As shown in Figure 3. the invention is based on the following principles : 

1. the synchronization - independently of the relative positions of the speaker 
(301) and auditors (300) - of the speaker workstation (306) and auditors 
devices (311) according to a the same universal time (312) (e.g., the GPS 
time, as provided by GPS receivers (309), (310), and 

2. the detection by means of a speech recognition system (305) operating on 
the speaker workstation (306) (with a microphone connected to it) of 
hyperlinked terms (304) (e.g., 'Wesi/era/ro/" (308)) pronounced by the speaker 

15 (301 ) during the course of the speech (302). 

Universal Timing Systems 

Common timing sequences, independent of locations of speakers and 
auditors, can be derived from an absolute timing reference such as. for example, the 
20 Global Posrtidning System (GPS) time or the Universal Time Co-ordinated (UTC) 
time (also known today as GMT and ZULU time). 

To transmit precise timing signals, the GPS uses 24 satellites in 55° inclined 
orbits 10.000 miles above the earth. These timing signals are used by any GPS 
receiver anywhere on the earth to determine its position. A 1575 MHz transmission 
25 canies a 1-IVIHz bandwidth phase-modulated signal named the clear acquisition 
(C/A) code. When a GPS receiver receives this signal from at least three GPS 
satellites, it can detemnine its own latitude and longitude with an accuracy of about 
30 meters. Apart the determination of geographical positions, the GPS is today 
widely used for distributing Precise Time and Time Interval (PTTI). The system uses 
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time of arrival (TOA) measurements for determining a position. A precisely timed 
clock is not essential for tlie user because time Is obtained in addition to position by 
tine measurement of tlie TOA of four satellites simultaneously in view. If the altitude 
over sea level is known, three satellites are sufficient. If the user is stationary at a 
5 known position then, in principle, the time can be obtained by the observation of a 
single satellite. Infomnation about the GPS time sen/ice is supplied by the "Time 
Sen^lce Department", U.S. Naval Obsen/atory, Washington, DC, at 
http://tyohoMsno,navy.mil/. 

10 GPS is today the world*s principal supplier of accurate time. It is used 

•extensively both as a source of time and as a means of transferring time from one 
location to another. Three kinds of time are available from GPS: GPS time, UTC as 
estimated and produced by the United States Naval Observatory, and the times from 
each free-running GPS satellite's atomic clock. The Master Control Station (MCS) at 

15 Falcon Air Force Base near Colorado Springs, Colorado, gathers the GPS satellites' 
data from five monitor stations around the world. A Kalman filter software program 
estimates the time error, frequency en-or, frequency drift and Keplerian orbit 
parameters for each of the satellites and its operating clock. This information is 
uploaded to each satellite so that it can be broadcast in real time. This process 

20 provides GPS time consistency across the constellation to within a small number of 
nanoseconds and accurate position determination of the satellites to within a few 
meters. 

The second universal time standard, Universal Time Co-ordinated (UTC), 
25 introduces leap seconds to remain synchronized with the rotation of the earth. In 
order to provide an estimate of UTC time derivable from a GPS signal, a set of UTC 
corrections is also provided as part of the GPS broadcast signal. This broadcast 
message includes the time difference in whole seconds between GPS time and 
UTC. This complicates software that deals with the smooth flow of data streams or 
30 calculates the times between data samples. GPS Time is preferred in this invention 
as this system avoids the introduction of leap seconds and is easily related to UTC. 
information about UTC (GMT) time sen^ice can be found on: 
http://tinie.greenwich2000.com/. 
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GPS receivers 

A Direct-to-Digital GPS Receiver is described in the following Web site : 
hnp'J/w3jesearch.ibm.com/present/gto200038Mm. 

This is an example of a tiny low cost chip which can Integrate GPS into anything 
5 (e.g., a PDA, a mobile phone, a wearable computer, a video camera) . This receiver 
has been jointly developed between IBM and Leica. The high speed analog 
capabilities of SiGe technology, when integrated with the CMOS technology, allows 
the integration of this single chip directly to a digital GPS (Global Positioning 
System) receiver. GPS derived position information is finding a multitude of diverse 
10 applications: from mapping and surveying to vehicle tracking to 91 1 cell phone caller 
location to automated farm equipment to even robotics golf carts... This receiver chip 
reduces the radio dimensions and complexity. There are no analog mixer stages, 
nor costly discrete components (such as high quality filters) that conventional two 
stage analog down conversion would have required. Instead, the incoming GPS 
15 signal is literally digitized right at the antenna, then filtered digitally in a CMOS based 
chip. This direct digitization is made possible by the ability of SiGe technology to run 
at high speed on very little power, and the core of this technology is a SiGe based 
Analog to Digital Data Converter. 



According to the present invention GPS or GLONASS receivers must be 



20 integrated or connected to the speaker workstation (typically a Personal Computer) 
and to auditor's devices (e.g., Personal Computers, wearable computers, Personal 
Digital Assistants (PDAs), smart phones or onboard mobile computers). The 
universal timing signals that are received from GPS or GLONASS satellites, are 
used to initialize and synchronize the internal electronic clocking systems on the 

25 speaker workstation and auditor's devices according to the same universal time. 
During the periods on which GPS or GLONASS satellites are out of sight (e.g., when 
speaker or auditor's devices are inside buildings or not connected to an external 
antenna), and no timing signals are thus received from those satellites, timing 
infomnation must be continuously derived from the autonomous electronic clocking 

30 systems. Depending on the drift of the internal clocking systems, and to keep 
enough timing precision and to be sure that user's devices are synchronized with 
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Speaker's workstations and broadcasting stations at tlie same universal time, a more 
or less frequent periodic reception of satellite signals must be perfonned. In practice, 

• If the user device is a portable or In-vehlcle mounted device, satellite signals will 
5 be received when the user is out of doors or is traveling. 

• If the user device is fixed or installed in a house or building during long periods, 
the user device must be connected to an outdoors installed GPS or GLONASS 
antenna, (e.g., antenna Installed on the roof of the building). 

Systems and Methods for Detecting Words in a Continuous Speech 

10 Speech recognition is the process by which a computer (or other type of 

system or apparatus) identifies spoken words. Basically, a speech recognition 
system is a computer which can correctly recognize what a speaker is saying. 

The speech recognition is an extremely difficult task. Unlike written text, no 
clear spacing exists between spoken words. Full sentences or sentences are 

15 typically pronounced without any pause. Furthemnore, the acoustic variability in the 
speech signal typically precludes an unambiguous mapping to a sequence of words 
or subword units, such as pronunciations of consonants and vowels. A major source 
of variability in speech is coarticulation, or the tendency for the acoustic 
characteristics of a given speech sound or phone sound to differ depending upon 

20 the phonetic context in which it is produced. 

Speech recognition systems can be categorized depending on the speaking 
styles, the vocabularies, and language models they accommodate. Isolated word 
recognizers require speakers to insert brief pauses between individual words. 
Continuous speech recognizers operate on fluent speech, but typically employ strict 
25 language models, or grammars, to limit the number of allowable word sequences. 

Word spotters are a particular kind of speech recognizers. They operate also 
on fluent speech. However, rather than providing a full transcription, word spotters 
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selectively locate relevant words or sentences. Word spotting Is useful to retrieve 
information based on keyword indexing or to recognize isolated words in voice 
command applications. 

Multiple speech recognition systems are today available with the word 
5 spotting capability required to support the present invention. These systems allow 
the detection of predefined words or sentences (hyperlinked tenns) that may be 
pronounced by a speaker in the course or a speech. Such speech recognition 
systems are, for instance, described in the following patents: 

US patent 5,199,077 entitled "Wordspotting for voice editing and indexing", 
10 discloses a technique for wordspotting based on hidden Markov models (HMM/s). 

The technique allows a speaker to specify keywords dynamically and to train the 

associated HMM's via a single repetition of a keyword. Non-keyword speech is. 

modeled using an HMM trained from a prerecorded sample of continuous speech. 

The wordspotter is intended for interactive applications, such as the editing of voice 
15 mail or mixed-media documents, and for keyword Indexing in single-speaker audio 

or video recordings. 

US patent 5,425,129, entitled "Method for word spotting in continuous 
speech", discloses a system and method for analyzing a digitized speech data 
channel for the presence of words or sentences from a desired list. The system and 
20 method of this invention, that has been implemented in the context of the IBM 
Continuous Speech Recognition System (ICSRS), provides in a speech recognition 
system a subsystem for spotting those pre-specified words or sentences in the 
presence of unrelated phonetic data. 

US patent 5,950,159 entitled "Word spotting using both filler and phone 
25 recognition" discloses a word-spotting system and a method for finding a keyword in 
acoustic data. The method includes a filler recognition phase and a keyword 
recognition phase wherein: during the filler recognition phase the acoustic data is 
processed to identify phones and to generate temporal delimiters and likelihood 
scores for the phones; during the keyword recognition phase, the acoustic data is 
30 processed to identify instances of a specified keyword including a sequence of 
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phones; wherein the temporal delimiters and likelihood scores generated in the filler 
recognition phase are used in the keyword recognition phase. 

US patent 6,006,185 entitled "System and device for advanced voice 
recognition word spotting** discloses a speaker independent, continuous speech, 
5 word spotting voice recognition system and method. The edges of phonemes in an 
utterance are quickly and accurately isolated. The utterance is broken into wave 
segments based upon the edges of the phonemes. A voice recognition engine is 
consulted multiple times for several wave segments and the results are analyzed to 
correctly identify the words in the utterance. 



10 US patent 6,073,095 entitled "Fast vocabulary independent method and 

apparatus for spotting words in speech" discloses a fast, vocabulary independent 
method, for spotting words in speech by utilizing a preprocessing step and a 
coarse-to-detailed search strategy for spotting a word/phone sequence in speech. 

US patent 6,185,527 entitled "System and method for automatic audio 
15 content analysis for word spotting, indexing, classification and retrieval" discloses a 
system and method for indexing an audio stream for subsequent information 
retrieval. 

US patent 6,230,126 entitled "Word-spotting speech recognition device and 
system" discloses a device for speech recognition which Includes a dictionary which 
20 stores features of recognition objects. The device further includes a matching unit 
which compares features of input speech with the features of the recognition 
objects, and a dictionary updating unit which updates time lengths of phonemics in 
the dictionary based on the input speech when the matching unit finds substantial 
similarities between the input speech and one of the recognition objects. 

25 A fundamental problem with speech recognition systems is the unreliable 

behavior of the state-of-the-art technology for performing "word spotting" (i.e., 
identification of pre-specified keywords or tehns) in a continuous mode, 
independently of the speaker, based on unknown or generic speaking styles, 
vocabularies, noise levels and language models. On the contrary, the automatic 
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word spotting function required for the present invention is relatively easy to 
implement since the system can be trained and tuned in advance by the speaker 
using his own voice, his specific speaking style and a specifically adapted 
vocabulary and language model. 

5 According to one aspect of the invention, a system and a method is disclosed 
for creating hyperlinks from pre-selected utterances (304) (commonly spoken words, 
tem^s or sentences) intended to be pronounced by a speaker (301) during a speech 
(302), to related relevant data (303) on the Web, and for training a speech 
recognition system (305) to identify (word spot) said hyperlinked terms (304). 

10 According to another aspect of the invention, a system and a method is 

disclosed for automatically recognizing (word spotting) the hyperlinked tenns (304) 
as they are pronounced by a speaker (301) during a live speech, and for creating on 
a network server a table comprising a list of recognized hyperlinked terms (304), the 
associated network addresses (i.e., the URLs) (303) and the universal times (309) 

15 corresponding to the recognition of these hyperlinked temis. 

According to a further aspect of the invention, a system and a method is 
disclosed for recording on an auditor device (311) universal times (310) 
corresponding to a selection (314) by an auditor. (300) during the course of a speech 
(302), of one or several interesting topics (313). 

20 According to another aspect of the invention, a system and a method is 
disclosed for enabling an auditor (300) to access and retrieve from a server 
connected to a computer networi<, relevant infonnation related to an hyperlinked 
term (304). 

Method for creating a Speech Hyperlink Table and for training a Speech 
25 Recognition System with the vocabulary of hyperlinked terms 

As shown In Figure 17, the invention discloses a system, method and 
computer program for use in by a speaker (301) before a speech, for creating 
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hyperlinks from selected utterances (304) (commonly spoken words or sentences) 
intended to be pronounced by a speaker (301) during a speech (302), to related 
relevant data (303) on one or several servers connected to a computer network 
preferably the Web, and for training a speech recognition system (305) for word 
5 spotting the speech with the vocabulary of said hyperlinked terms (304). The method 
compnses the steps of: . 

during the production or edition of the speech: 

• (1 701 ) editing the text or a draft text (400) of the speech; 

• (1702) selecting and marking on the speech text (500) a plurality of relevant 
10 terms, or words (501 ) for which hyperlinks must be created; 

• (1703) creating a Speech Hyperlink Table (600) for the speech; 

• defining hyperlinks in the Speech Hyperlink Table (600) between the selected 
terms or words (501) and multimedia information or services located on one or a 
plurality of servers (909) connected to a computer network (908), said step 

15 comprising the further steps of: 

for each one of the selected hyperlinked terms or words (501): 

• (1704) assigning a name and/or a description (601) (preferably a brief 
description); 

• (1705) assigning a destination address (602) (for instance the URL) within the 
20 network (908) to access the desired multimedia infomnation or sen/ice; 

• (1706) storing in the Speech Hyperlink Table (600) the assigned name (or 
description) (601 ) and/or the destination address (602); and 

once hyperlinks have been defined in the Speech Hyperlink Table (600): 

• (1707) training a speech recognition system (701) running on the speaker 
25 workstation (702) with the vocabulary of the hyperlinked terms (602), (703) for 

automatically recognizing these hyperlinked tenms when they will be pronounced 
by the speaker (700) during the speech. 
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Figure 4 shows the text (400) of an exemplary speech (e.g., a conference 
about the subject "Wine & Healtif), to be perforrri in a conference room or to be 
broadcast at the radio or television. During the live speech, this text can be read in 
its entirety by the speaker (301) or the speal<er can intentionally omit to read of 
5 some parts, change the order, or introduce alternative or additional comments at 
convenience. 

Figure 5 illustrates how a speaker, a program editor (or any person committed 
do to it) takes the speech text (500) and selects certain terms (501) (words or 
sentences, such as "Phenols", "Resveratrol", "Tannins", "Ellagic acid", 
10 "Hydroxycinnamates", "Free radicals", "Antt^ocyanin", "Gallic acid" ) to associate 
additional information accessible on the Web. Each of these selected terms named 
hyperlinked temns, must be associated to a network address (i.e., to the 
corresponding URL) on the Web from where the related information or sen/ice can 
be retrieved. 

15 Figure 6 shows how the speaker, the program editor (or any person 

committed do to it) creates on the speaker workstation (306) a Speech Hyperlink 
Table (600) that associates with each selected hyperiinked term (601) (i.e., selected 
word or sentence on the speech text, e.g., "Resveratrol"), a corresponding URL on 
the Web (602) (e.g., httpy/www.ag.uiuc.edu/-ffh/resvera.html) 

20 Figure 7. shows how. before the speech, the speaker (700) trains with the 
vocabulary of hyperlinked terms (703) (e.g.. "Phenols", "Resveratrol", Tennis", ... ) a 
speech recognition system (701) with word spotting capability installed on his 
workstation (702). In a particular embodiment of the invention, the word spotting 
function is implemented by means of the IBM Continuous Speech Recognition 

25 Sysfem - ICSRS - running.on the IBM ViaVoice software product. 



Method for recognizing hyperlinked terms during a speech and for creating a 
Speech Hyperlink-Time table on a Speech Server 
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As Shown in Figure 18, the invention also discloses a system, method and 
computer program for use in a spealcer worl<station (802) for recognizing 
hyperlinked temis (803) as they are pronounced by the speaker (800) during a live 
speech, for creating on a Speech Server (907) connected to a network (908). a 
Speech Hypeilink-Time Table (906). and for updating this table with records 
comprising the sequence (905) of the recognized hyperlinked terms (304), the 
corresponding networt< addresses (i.e., the URLs) (303) and the universal times 
(309) when these hyperiinked temis have been recognized. IVIore particularly, the 
method comprises the steps of: 

• (1801) creating a Speech Hyperlink-Time Table (906) for the speech on a 
Speech Server (907) connected to a network (908); and 

during the speech: 

• (1802) Vord spotting" the speech to recognize the pronunciation by the speaker 
(800) of hyperlinked temis (803) by means of a speech recognition system (801) 
running on the speaker wori<station (802) and purposely trained for the speech; 
and; 



for each hyperlinked tenn (803) that has been recognized: 

• (1803) detemiining by means of an universal time device, the univer^l time 
(904) corresponding to the recognition of the hyperiinked term (903); 
20 • (1804) creating a record including: 

• the universal time (904) when the hyperiinked term (903) has been 
recognized; 

• the name or short description (601) of the recognized hyperiinked term 
(803) using the Speech Hyperiink Table (600); and 

• the networi< address (1005) (i.e., the URL) corresponding to the 
recognized hyperiinked temi (803) using the Speech Hyperiink Table 
(600); 



25 
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(1805) Storing the record in the Speech Hyperiink-Time Table (906), (1000) 
stored on the Speech Server (907) accessible through the networi< (908) from 
the speal<er workstation (902). 



Figure 8 illustrates how, while the speaker (800) is pronouncing his speech, 
5 the "word spotting" function running on the speech recognition system (801 ) installed 
on the speaker workstation (802) (e.g., implemented by means of the IBM 
Continuous Speech Recognition System - ICSRS - running on the IBM ViaVoice 
software product) and previously trained by the speaker, automatically detects the 
hyperlinked tenns (803) from the voice stream. 

10 Figure 9 shows how, after recognition of the hyperlinked term (903) (e.g., 

"Resveratrol') by means of the speech recognition system (901) running on the 
speaker workstation (902), 

• the universal time (904) (e.g., 12/05/2001 14:23:18) corresponding to the 
recognition of the hyperiinked term (903); 

15 • the hyperlinked term name or short description (601) (e.g., "Resveratrol^; and 

• the associated URL (602) (e.g., http://www.ag.uiuc.eclu/~ffh/resvera.html); 

(last both extracted from the Speech Hyperiink Table (600) are sent (905) and 
stored on a Speech Hyperiink-Time Table (906) located on a Speech Server 
(907). 

20 Figure 1 0 shows an example of a Speech Hyperiink-Time Table (1 000) (906) 

stored on the Speech Server (907) as it appears at the end of the speech (e.g., 
about the subject 'Wine & Health" (^00^)). The header of this table comprises the 
URL (1002): http://www.dlrectbuyer.com/conference-0173.htm/or networi< address 
where the Speech Hyperiink-Time Table (1000) has been stored in the Speech 

25 Server (907) {e.g., www.directbuyer.com). The URL where the Speech 
Hyperiink-Time Table (1000) can be found in the networi<, must be known in 
advance by the auditors. Each row in this table corresponds to a hyperiinked term 
(903) pronounced by the speaker and recognized by the speech recognition system 
(901) during the speech. Each column corresponds, respectively to: 
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• (1003) the universal time corresponding to the recognition of a spol<en 
hyperiinked term (903) by the Speech Recognition System (901) during the 
speech; 

• (1 004) the name and/or short description (601 ) of the recognized hyperlinl<ed 
(903) tenrj copied from the Speech Hyperlink Table (600); 

• (1005) the URL (602) con-esponding to the recognized hyperiinked term 
(903). copied from the Speech Hyperiink Table (600). 

Method for creating a Selections Hyperlink-Time Table and for selecting topics 
of interest 



10 



As illustrated by Figure 19, the invention discloses also a system, method and 
cdmputer program for creating a Selections Hyperiink-Time Table (1106) on an 
auditor device (1102) and for recording in this table a sequence of universal times 
(1105) corresponding to the selection (1104) by an auditor (1100) of interesting 
15 topics (1 1 03) during the speech, for the purpose of, immediately after the speech or 
at a later time, receiving further infomiation related to these topics. More particulariy. 
the method for use in the auditor device, comprises the steps of: 

• (1901) creating a Selections Hyperiink-Time Table (1200) on the auditor device 
(1102); 

20 . (1902) recording on the Selections Hyperiink-Time Table (1200) the network 
address (i.e., the URL) (1201) of the Speech Hyperiink-Time Table (906) stored 
on the Speech Server (907); 

• (1 903) listening to the speech (1 1 07) pronounced by the speaker (1101); 

• (1904) perceiving in the speech, a topic of interest (1103) for which additional 
25 infonnation or sewice is desired; 

• (1905) selecting this topic of interest (1103) by entering a selection command 
(1 1 04) on the auditor device (1 1 02); 

• (1906) determining the current universal time (1105) by means of an universal 
time device (for instance a GPS receiver) integrated or connected to the auditor 

30 device; 
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• (1907) recording this current universal time (1105) in tlie Selections 
Hyperlink-Time Table (1106). (1202). 

Figure 11 illustrates how during the speech, the auditor (1100) operates using 
a portable auditor device (1102) in stand-alone mode (i.e., isolated, not connected to 

5 a network). According to the particular embodiment detailed in Figure 11, each time 
the auditor (1100) perceives during the speech an interesting topic (1103), he 
immediately selects this topic simply by pressing a reserved, key (1104) on the 
auditor device (1102). Universal times (1105) corresponding to the selection of the 
topics by the auditor, are stored in the Selections Hyperlink-Time Table (1106) on 

10 the auditor device (1 1 02). 

Figure 12 shows a typical Selections Hyperlink-Time Table (1200) created on 
the auditor device (1 102) during the speech. Each row in this table corresponds to a 
different universal time corresponding to the selection by the auditor of an interesting 
topic during the speech. The header of this table comprises the URL (1201) (e.g., 

15 http://www.directbuyer.conVconference-0U3M^ ) of the Speech Hyperlink-Time 
Table (906),(1000) corresponding to the speech. The Speech Hyperlink-Time Table 
(906),(1000) on the Speech Server (907) (e.g., ymwMirectbuyer.cow), is updated 
during the speech from the speaker workstation (902). As it has been said before, 
the URL of the Speech Sender must be provided in advance to the auditors to allow 

20 them to create locally a Selections Hyperlink-Time Table (1200). 



Method for retrieving hyperlinks and for accessing information 

As illustrated by Figure 20, the invention discloses also a system, method and 
computer program for enabling an auditor (300) to access and retrieve from the 
Web relevant data or infonnation related with the hyperlinked terms (304) that were 
25 active when the auditor made the con'esponding selections (314). More particularly, 
the method for use on the auditor device (1300), comprises the steps of: 



• (2001 ) accessing the Speech Server (1303); 
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for each universal time (1202) recorded in the Selections Hyperlink-Time Table 
(1304), (1200) located on the auditor device (1300): 

• (2002) sending (1306) to the Speech Sender (1303) the recorded universal 
time; 

5 • (2003) identifying In the Speech Hyperlink-Time Table (1302), (1000) located 
on the Speech Server (1303), the hyperlinked temn (1004) associated with 
said universal time (1306) (hyperlinked term that was active at that universal 
time); 

• (2004) retrieving (1307) from the Speech Hyperlink-Time Table (1302), (1000) 
10 located in the Speech Server (1303) the name (or description) (1004) and the 

destination address (URL) (1005) of the selected hyperlinked term; 

• (2005) storing the retrieved hyperlink name and destination address (URL) 
(1307) into the Selections Hyperlink-Time Table (1304), (1402) located on the 
auditor device (1300); 

15 

The method includes the further steps of: 

• (2006) selecting in the Selections Hyperlink-Time Table (1502). an hyperlink 
(1501) using the hyperlink name (or description) or the associated destination 
address retrieved from the Speech Server (1507); 

20 • (2007) activating said hyperlink (1501) using a browser program running on the 
auditor device (1500); 

• (2008) accessing the infomiation and/or sen^ice located on a server (1506) 
connected to the network (1505) using the retrieved destination address (1503) 
(1504) associated with the selected hyperlink (1501); 

25 • (2009) retrieving (1601) the infomiation and/or service (1604) from the accessed 
server (1603) on the network (1602); 

• displaying, on the auditor device (1600), the retrieved infomiation and/or service 
(1 604) using a browser program 

Figure 13 illustrates how, by connecting the portable device (1300) of the 
30 auditor (1 1 00) to the communications networi< (1 301 ) (e.g., the Internet networi<) and 
by accessing the Speech Hyperiink-Tlme Table (1302) on the Speech Server (1303), 
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the information on the Selections Hyperlink-Time Table (1304) of the auditor device 
(1300) is updated. 

Figure 14 illustrates in detail the process for updating the Selections 
5 Hyperlink-Time Table (1400) on the auditor device (1300) using the Speech 
Hyperlink-Time. Table (1401), (1302) on the Speech Server (1303). Basically, by 
means of this process, the hyperlinked terms (or short names and/or short 
descriptions) and the URLs of the hyperlinked \erms that were active at the 
universal times (1 403) when the auditor has selected interesting topics, are identified 
10 in the Speech Hyperlink^Time Table (1401) located on the Speech Server (1303) 
and are copied into the Selections Hyperlink-Time Table (1402) located on the 
auditor device (1300). 

Figure 15 illustrates how the auditor selects and get access from the auditor 
device (1500) to Web pages hyperlinked to the topics he has selected during the 

15 speech. Basically, the auditor points to and selects a hyperlinked term (1501) on the 
updated Selections Hyperlink-Time Table (1502) and, by means of a software on the 
auditor device (1500), activates a Web browser and triggers the hyperlink to the URL 
(1503) of the selected item (1501). In the example shown in this figure, the auditor 
selects the hyperlinked term "Ellagb acid", (in fact a chemical compound found in 

20 wines), and triggers the hyperiink pointing to the URL (1 504): 
http://wvmMopeforcancer.com/ellagicacid.htm 

Figure 16 shows how the Web page (1601), (e.g., document: ellagicacid.htm). 
associated with the selected hyperiinked temfi (1501), (e.g., "Ellagic acid"), is 
received through the networic (1602) from the accessed Web server (1603) (e.g., 
25 http://wvm.hopeforcancer.com), and displayed (1604) or played on the auditor 
device (1600). 

While the invention has been particularly shown and described with reference 
to a preferred embodiment, it will be understood that various changes in fonn and 
detail may. be made therein without departing from the spirit, and scope of the 
30 invention. 
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Claims 



What is claimed is : 

1 . A method, for generating from a speaker device, a Speech Hyperlink-Time table 
during a speech, said Speech Hyperlink-Time table being accessible by one or a 
5 plurality of auditors, comprising the steps of: 

• locating and accessing a Speech Hyperlink table comprising : 

• means for identifying one or a plurality of predefined hyperlinked terms 
intended to be pronounced by a speaker; 

• means for locating and accessing information related to each of said one or 
10 plurality of predefined hyperlinked terms; 

• retrieving means for locating and accessing a Speech Hyperlink-Time table; 

during the speech : 

• recognizing, when they are pronounced by the speaker, the hyperlinked temis 
predefined in the Speech Hyperlink table by means of a speech recognition 

15 system connected to the speaker device; 

for each recognized hyperlinked temi : 

• detemnining an universal-time corresponding to the pronunciation by the speaker 
of the recognized hyperlinked term; 

• creating a new record on the Speech Hyperlink-Time table; 
20 • copying into said new record : 

• the universal-time corresponding pronunciation by the speaker of the 
recognized hyperlinked term; 

• means, retrieved from the Speech Hyperlink table, for identifying the 
recognized hyperlinked term; 

25 • means, retrieved from the Speech Hyperlink table, for locating and accessing 



information related to the recognized hyperlinked tem. 
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2. The method according to the preceding claim comprising the preliminary steps of : 

• creating a Speech Hyperlink table, preferably in the speaker device; 

• defining, in said Speech Hypertirik table, hyperlinks to one or a plurality of 
5 servers where information related to predefined temns can be accessed. 

3. The method according to the preceding claim wherein the step of defining, in said 
Speech Hyperlink table, hyperlinks to one or a plurality of servers where information . 
related to predefined hyperlinked terms can be accessed, comprises the further 
steps of: 

10 • predefining hyperlinked terms intended to be pronounced by the speaker during 
a speech; 

and, for each new predefined hyperlinked term : 

• creating a new record in the Speech Hyperlink table; 

• assigning and copying means for identifying the predefined hyperlinked term in 
15 said record; 

• assigning and copying means for locating and accessing information related to 
the predefined hyperlinked term in said record. 

4. The method according to any one of claims 2 to 3 wherein the step of creating a 
Speech Hyperlink table, preferably in the speaker device, comprises the further 

20 steps of: 

• assigning to said Speech Hyperlink table means for locating and accessing the 
Speech Hyperlink-Time table; 

• copying said locating and accessing means in the Speech Hyperlink table, 
preferably in a header. 

25 5. The method according to any one of the preceding claims wherein the step of 
recognizing, when they are pronounced by the speaker, the hyperlinked tenns 
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predefined in the Speecli Hyperlink table by means of a speech recognition system 
connected to the speaker device, comprises the preliminary step, before the speech, 
of : 

• training the speech recognition system to recognize the predefined hyperlinked 
5 tenms when they are pronounced by the speaker. 

6. The method according to any one of the preceding claims wherein said speech is 
a television or radio broadcast program. 

7. The method according to any one of the preceding claims wherein : 

• said information related to a predefined hyperlinked term is located on one or a 
10 plurality of sen/ers; 

• said means for identifying a predefined hyperlinked temi comprises a name 
and/or a description; 

• said means for locating and accessing information related to a predefined 
hyperlinked term comprises a destination address within a network; 

15 • said Speech Hyperiink-Time table is located on a Speech Server; 

• said means for locating and accessing the Speech Hyperiink-Time table 
comprises a destination address within a network, 

8. The method according to any one of the preceding claims wherein : 

• said networi< is an internet network; 

20 • said destination addresses are Uniform Resource Locators; 

• infomiation related to a predefined term is. a web page; 

• said servers are web sen/ers. 

9. The method according to any one of the preceding claims comprising the step of: 

• sending to one or a plurality of auditor devices means for locating and accessing 
25 the Speech Hyperiink-Time table. 
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10. A speaker device comprising means adapted for carrying out the method 
according to any one of the precedmg claims. 

11. A computer program comprising instructions for carrying out the method 
according to any one of claims 1 to 8 when executed on the speaker device 

5 according to the preceding claim. 

12. A method for selecting from an auditor device, one or a plurality of hyperiinked 
terms pronounced by a speaker during the course of a speech, and for accessing, at 
a later time, information related to each of said one or plurality of selected 
hyperiinked terms, comprising the steps of : ' 

10 • each time a selection command is received for selecting a term currently 
pronounced by a speaker during the course of a speech: 

• determining the current universal-time; 

• recording the current universal-time in a Selections Hyperiink-Time table; 

• accessing a Speech Hyperiink-Time table, said Speech Hyperiink-Time table, 
15 accessible by one or a plurality of auditor devices, comprising for each one of a 

plurality of predefined hyperiinked temrts pronounced by the speaker : 

• an universal-time interval corresponding to the pronunciation of the 
predefined hyperiinked term; 

• means for identifying the predefined hyperiinked term; 

20 • means for locating and accessing information related to the predefined 
hyperiinked term; 

for each universal-time recorded in the Selections Hyperiink-Time table: 

• identifying in the Speech Hyperiink-Time table, the selected hyperiinked term 
pronounced at the recorded universal-time; 

25 • retrieving from the Speech Hyperiink-Time table, said means for identifying and 
for locating and accessing the selected hyperiinked term; 

• storing the retrieved means for identifying the selected hyperiinked term and for 
locating and accessing information related to the selected hyperiinked term into 
the Selections Hyperiink-Time table. 
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13. The method according to the preceding claim comprising the preliminary step of: 
• creating the Selections Hyperlinl<-Time table for the speech. 

14. The method according to any one of claims 12 to 13 comprising the preliminary 
step of: 

5 • receiving means for locating and accessing a Speech Hyperlink-Time table. 

15. The method according to any one of claims 12 to 14 wherein : 

• information related to a hyperlinked term is located on one or a plurality of 
servers; 

• said means for identifying a hyperlinked term comprises a name and/or a 
10 description; 

• said means for locating and accessing information related to a hyperiinked term 
comprises a destination address within a network; 

• said Speech Hyperlink-Time table is located on a Speech Server (803); 

• said means for locating and accessing the Speech Hyperlink-Time Table 
15 comprises a destination address within a networi<. 

16. The method according to any one of claims 12 to 15 wherein : 

• said network is an internet networ1<; 

• said destination addresses are Uniform Resource Locators; 

• infonmation related to a term is a web page; 
20 • said auditor device comprises a web browser; 

• said servers are web sen/ers. 



17. The method according to any one of claims 12 to 16 comprising the further step 
of: 
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• receiving a display command for displaying infomiation related to a hyperlinl<ed 
tenn, said display command comprising means for Identlf^ng said hyperlinl<ed 
tern; 

• identifying, In the Selections Hyperlink-Tlme table, means for locating and 
5 accessing infonmation related to said hyperlinked tenr^; 

• retrieving and displaying said infomiation. 

18. The method aqcording to any one of claims. 12 to 17 wherein said speech is a 
television or radio broadcast program. 

19. An auditor device comprising means adapted for canying out the method 
10 according to any one of claims 12 to 1 8. 

20. A computer program comprising instructions for carrying out the method 
according to any one of claims 12 to 18. when executed on the auditor device 
according to the preceding claim. 




Fig. 1: During a speech auditors perceive interesting topics 




Fig. 2: During a radio or television program an auditor 
perceives interesting topics 



wo 2004/029831 PCT/EP2003/009310 

3/20 



(304) 



(303) 



Hypeiflnked term i 


URL y 


"Phenow / 


htlp:/Mww.ston8hil^MnoIy.CTfn/heaWl.ht^ / 


"Resvaratrar / 






httprih«iww.inangtarebm«.n6ViMnemttnw 


"Ellaglc add" 


htlp7Art^w.hop«fbiwncer.com/6llaglcackLMm 


"Hydioxydnnamatesr 


http'^AMWW.wrats.nltepa/ioumata/^te 


'Ravon-3-ols" 


http-jMwwjiyas.«s^K«sclitern^ 


"Fcee radicals" 


http'7Aivvw.who.acleg/publleatlon&/tai^ 


"Anthocyanln" 




"Game add' 





(308) 

.jfiesveratrol 
loses glucose of 
its molecule ... 



(302) 



Speaker 
(301) 



Speech Recognition System 
(305) 

Speaker 
workstation 
(306) 

(309 



Synchronized universal time 
(312) 



microphone 
(307) 



Auditor device 
(311) 



Principles of the 
Invention 




Fig. 3: The speaker workstation and auditors devices are 
synchronized. Hyperiinked terms pronounced by the 
speaker are automatically recognized during the speech 
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,(400) 



The concentration and molecular stmcture of the phenols changes as 
grapes are processed into wine, and as new wine ages Into old wine. 
Oxidation (the addition of an oxygen atom to a molecule) taices place, 
and the resulting molecules further interact with each other. 
Resveratrol loses glucose of its molecule. The.oak banrei tannins 
break down and release ellagic acid. After a few months, the 
hydroxycinnamates begin to break down. The "flavon-3-olsfrom 
bonds with other molecules and change Into a form that is not found 
in the fresh grape. 

The good news about all these chemical changes is that as wine ages, 
the number of anti-oxidizing molecules in it actually increases. 
Anti-oxidants purge the body of free radicals and are good for us. 

The bad news is that, despite the increase in the number of 
anti-oxidizing molecules overtime, their bioavailability decreases. 
The anti-oxidizing molecules that are formed during aging are much 
larger than the original phenols, and thus are not well absorbed in the 
alimentary canal. After one or two years of aging, the amount of 
anthocyanin has decreased to about one-sixth of the original level. 
After five to ten years, wine has lost most of the smaller, and 
presumably health friendly, phenols. "When we look at very old wine, 
the only component that stands out is gallic acid". 

So, ironically, a $7 bottle of 3-year old wine Is a healthier drink than a 
$600 bottle of 26.year old wine. (Of course, it may not taste as goodi). 



Affi Wmt loses Its Vitality, ly: Andrew L Watcrhmise, PhlX VitintlttiTe and Etu^ at the University (f 

CaUJomia at Davis Cotfcrenct on AIM and Whu in Health mdDisease, New Yori Acaimt tf Sciences AM 
26-29,2001 JJ>f 

bttp./ /vMw.ryasMg/scitedi/sum/anfjO l_0426J)tml# 13 



Fig. 4: The speaker prepares the speech text 
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(501) 



The concentration and mrtc^curar iatrl^ the phenols changes as 
grapes are processed IritO A^ne^ r^w wln^ ages Into old wine. 
Oxidation (the.atfdition .of an o^^en jatom io a motequle) takes place, 
and the fesalfing mpl^cules 1)lrtfier ii||eract ^th each isther. 
Resvefatrol iosejs glucose of molecule. The pak barrel fainnins 
br§Bk down and release ell4g^c acid . Iftfter a few months, the ' 
hvdroxvcinriamates begin to break dbW The flaVon-3^ls from 
bonds yiMh other molecMles and charge into a fomi^t is not found 
in the ifiresh grape. h 



The good news abaut all these chemi^l changes is that as wine ages, 
the number of anf|4xldizing molecule^ )n it actually Increases. 
Anti-oxidants puj^ the body of free radicals and are good for us. 

The bad new^^s that, despite the incr4a$e in the number of 
anti-oxidizin0 molecules overtime, thdir bioavailability decreases. 
The anti-o)^clizing molecules that are for^d during aging are much 
larger than the original phenols, and thu^ are not well absorbed in the 
aliment^iy canal. After one or two years ^f aging, the amount of 
anthocyanin has decreased to about onO'lsixth of the original level. 
After five to ten years, vnne has lost most bf the smaller, and 
prei^mably health friendly, phenols. "Wh4n we look at very old wine, 
th^^niy component that stands out is gallic acid" . 



$0, ironically, a $7 bottle of 3-year old win^ is a healthier drink than a 
$600 bottle of 25^r old wine. (Of courae^lt may not taste as goodl). 

I 



Agci Wim Uses Its Vitality, by: Andrew L Wattrlmsc, 2%X>. Dcp,of Vitiailturc and Bw% at the Unrmsity of 
^eisVoQl^ ^^^^^ ^ ^^^^ ^ ^ W JDR«a», New York Acaimy <f Sciences, A^ril 
ktp:/ 1 'miw.rtyas.org/sciiuh/ sm/coi^J) lj)i2 6,htmM 13 



Fig. 5: The speaker selects hyperlinked terms in tlie text 
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Speech Hyperlink Table (6oo) 



II ■■ I ■ 

Hyperlinked 
term 


URL / 


"Phenols" 


http.7Avww.stonehlllwlnefy.com/health.html 


"Resveratrol" 


http://www.ag^iuc.edu/'-ffh/resvera.html 


'Tannins" / 


http://www.mangiarel)ene.net/Wlne/manual/health.html 


"Ellagicadd" 


http://Vvww.liopeforcancer.com/ellagicacid.htm 


"Hydroxypinnamates" 


httpV/Vvwvy.swets.nl/sps/joumaIs/pl)36000s.html 


"Flavon/3-ols" 


http://WvyW.nyas.org/scitech/sum/conf_01_0426.htmi 


"Free jf^dicals" 


http://v^.who.sci.eg/r)ublications/emhj/0402/21.htm 


"Antfciocyanin" 


http:/|www.orst.edii/dept/lpi/ss01/anthocyanin.html 


"Gallic acid" 


httpi?i!/www.vinsetsante.com/za5e.html 


(m 


(602) 



Fig. 6: The speaker creates a Speech Hyperlink Table and 
associates an address with each hyperlinked term 
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(703) 

Vocabulary of 
hyperlinked terms 

"Phenols" 
"Resveratrol" 



"Tannins" 



"Ellagicacid" 



"Hydroxycinnamates" 



"Flavon-3-ols" 



"Free radicals" 



"Anthocyanin" 



"Gallic acid" 



Speaker 
(700) 




Speaker 
workstation 
(702) 



Speech Recognition System 
(701) 



Fig. 7: The speaker trains the Speech Recognition 
System to recognize hyperlinked terms 
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(803) 

Vocabulary of 
hyperlinked terms 



"Phenols" 



"Resveratrol" 



"Tannins" 



•Ellagicacid" 



"Hydroxycinnamates" 



"Flavon-3-ols" 



"Free radicals" 



"Anthocyanin" 



"Gallic acid" 



(803) 



Speaker 
(800) 




workstation 
(802) 



Speech Recognition System 
(801) 




Fig. 8: During the speech, the Speech Recognition System 
recognizes hyperlinked terms pronounced by the speaker 
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Speech Server 
(www.directbuyer.com) 
(907) 




(^12^05/2001 14;23;18 "Resveratrol" http;//www.aa.i 



Universal time of 
hyperiinlced term 
recognition 
(904) 

( 12/05/2001 \4:23:18)1 

\ 




Speecfi Recognition 
(901) 



Fig. 9: Hyperiinl^ed terms and associated addresses and universal 
times are recorded on a Speech Hyperiinlc-Time Table on a 
Speech Server 
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(1001) 



Speech Hyperlink-Time Table 



(1002) 



SPteCH TITLE: 
l/Vine & HeaRhj 


SPEAKER: 


f^pppf^u uYPFRliNK-TIMFTARI F URI ' / 


Louis Durand ^ 


T^fipiy/www.direclbuy 


RECOGNITION 
TIME 


HYPERLINKED 
TERM 


HYPERLINK URL 


12/05/2001 14:22:52 


Phenols. 


http://vvww.stonehlllwinery.com/health.Mml 


12/05/2001 14:23:12 


Resveratrol 


http://www.ag.uiuc.edu/-^ffh/resvera.html 


12/0£|/2001 14:2325 


Tann^ 

\ 


http://www.mangiarebene.net/wine/mabual/health.htm 

' . \ 


12/05/^001 14:23:39 


Ellagid.acid 


http://www.hopeforcancer.com/ellagicac^d.htm 


12/05/2001 14:23:57 


Hydroxyclnnamates 


http-7/Vvww.swets.ni/sps/joumals/pb360o4s.html 


12/05/2001 1424:11 


Flavon-S-ols 


http://www.nyas.oig/scitech/sum/conL0li0426.html 


12/05/20)1 14:24:35 


Free radicals 


http://www.who.sci.eg/publications/emhj/0402/21.htm 


12/05/20Q1 14:24:49 


Anthocyabin 


http://www.orst.edu/dept/lpi/ss01/anthocyafiin.html 


12/05/20ti|l 14:25:08 


Gallic acid. 


http://www.vinsetsante.com/za5e.html 




\ 
\ 


\ 




•\ 




\ 

— » 










^ 




\ 


\ 



(1000) (1003) 



(1004) 



(1005) 



Fig. 10: 
Speech 



Example of Speech Hyperlink-Time Table stored on the 
Server 




Fig. 11: During the speecii, the auditor selects interesting 
hyperlinlced terms by pressing a reserved key on the auditor device 
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[1202) 



Selections Hyperlink-Time Table 



(1201) 





SPEAKER* 

Louis Durand 


OlfUwflflTI'cKuNfV-TI / 

;*fittp:/AA^ 


.SELECTION TIME , 


HYPERLINK 


HYPERLINK URI 


12/05/2001 14:23:14 






12/05C001 1423:42 






.12/05e001 1424:52/ 


































































/ 







(1200) 



Fig. 12: Universal times corresponding to the selection of 
terms by the auditor are stored in a Selections 
Hyperiinl(-Time Table on the auditor device 
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Speech Server 
(1303) 




Speech 
Hyperlink-Time Table 
(1302) 



"Riesveratrol", 

http://www.ag.uiuc.edii/~ffh/resvera.html 



( 12/05/2001 14:23:14) 



(1307) 



Selections 
Hyperlink-Time Table 
(1304) 




Auditor device 
(1300) 



Fig. 13: The auditor updates the Selections Hyperlink-Time 
Table from the Speech Hyperlink-Time Table stored on the 
Speech Server 
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SPEECH TITLE: 

vvine cx ricaiin 


SPEAKER: 
Louis Durand 


SPEECH HYPERLINK-TIIWETABLE URL: 

http://iAAftW.cfirectbuyer.com/conference^ 


) 


SEtfCtlONtlME... 


HYPERUNK 


HYPERLINK URL 


12/05/2001 14:23:14 








12/05/2001 14:23:42 


} 

/ 






12/05/2001 14:24:52. 

' ■' - — C 






N 


1403) 


(1400 


SPEECH TITLE: 
Wine & Health 


SPEAKER: 
Louis Durand 


SPEECH HYPERLINK-TIMETABLE URL: 

Mlp:/Aflww.dlrectbuyer.corn/conference-0173.htm/ 


RECOGNITION 
TIME 


HYPERLINKED 
TERM 


HYPERLINK URL 


12/06/2001 14:22:52 


Phenols 


httpi/Aww.stonehillvvinery.com/health 


12/05/2001 14:23:12 


Resveratrol 


http:/AMAw.ag.uiuc.edu/-ffh/resvera.html 






12/06/2001 14:23:25 


Tannins 


http:/Awvw.mangIarebene.netAMne/maniial/health.ht^ 


12/05/2001 14:23:39 


Ellagic acid 


http://vwvw.hopeforcancerxom/eIIagicacid;htm 






12/05/2001 14:23:57 


Hydroxycinnamates 


http://wvvw.swets.n!/sps/(ournals/pb36000s.h^^^ 


12/05/2001 14:24:11 


Flavon-3-ols 


http:/Mww.nyas.org/scitech/sum/a)nfJ)1_^ 


1^05/2001 14:24:35 


Free radicals 


http:/^vww.who.sd.eg^publlcations/em 




12/05/2001 1454:49 


Anthocyanin 


http7A^w.oret.edu^depl/lpi/ss01/anthocyaniahtml 


) 




12/05Q001 1425:08 


Gallic acid 


http:/Aw(vw.vlnsetsante.com/za5e.htmi 












1 




(1401 


SPEECH TITLE: 
Wine & Health 


SPEAKER: 
Louis Durand 


SPEECH HYPERUNK-TIMETABLE URL: 

http7/www.directbuyer.com/conference-0173.htm/ 


SELECTION TIME 


HYPERLINK 


HYPERUNK URL 


12/05/2001 14:23:14 


Resveratrol 


http://vvvvw.ag.uiuc.8du^Mfh/resvera.htrnl 


12/06/2001 14:23:42 
12/05/2001 14:24:52 


Ellagic acid 
Anthocyanin 


http://www,hopeforcancer.com/ellagicacldhtm 



(1402) 



Fig. 14: Hyperlinked terms selected by the auditor are identified 
and copied from the Speech Hyperlink-Time Table Into the 
Selections Hyperlink-Time Table 



) ) 
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www.hopefbrcancer, 




Speech Server 
(1507) 



"EUagic acid" 
hyperlink at: 




(1^4) 



(|^http://www.hopefforcancerxom/ellagicacid.htfii^ 




Selections 
Hyperlinl(-Tlme 
Table 
(1502) 



Auditor device 
(1500) 




Fig. 15: Hyperlinlcs are triggered using the Selections 
Hyperlink-Time Table 
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WWW. 



iopeforca ncer.com 

Web Servers 
(1603) 



Speech Server 




Retrieved Web 




EUAQICACID ^ 

Ellagic add Is a phenolic compound found In plants in the form of 
h/drolyzabie tannins called eltagitannlns. ERagltanNns are esteis 
of gtucose with hexahydraxydlphenic add; when h/dfDlyze(l, they 
yield Ellagic acid, the dllactone of he)ahydta)vdiphenlc add 
Blagic add Is a veiy stable compound and is readU/ absorbed 
through the gastrdrttestlnat system in mamm^, Indudlr^ huirans. 

Berries are the most common ttood sources of EOagic add The 
relatiVB amount of Blagic add In average number of micrograms 



Received 
Web page 
(1604) 



Auditor device 
(1600) 





Fig. 16: The Web page related to the selected term is received 
from the Web server and displayed on the auditor device 
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Editing the text or a draft text of the speech 

\ ~ 



-^1701 



Selecting on the speech text a plurality of wordi 
or terms for which hyperlinis must be created 



1702 



^ /1703 



Creating a Speech Hyperlink Table for the speech^ ^ 



on the speaker workstation 



Assigning a name or a description (preferably 
a brief description) to hyperlinked terms 



I 



1705 



Assigning a destination address (URL) 
to access multimedia information 

* ^ — • 



Storing in the Speech IHyperiink Table the assigned: ^ 
' hyperiink name (or description), and 
- the destination address 



1706 



I 



Training with the hyperiinked temis 

a Speech Recognition System 
running on the speaker woricstation 



.1707 



Fig. 17: Method for creating a Speech Hyperlink Table for a 
speech and for training a Speech Recognition System with 
hyperlinked terms 
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Creating a Speech Hyperlink-Time Table 
for the speech on a Speech Server 



1801 



. — L : ^^B02 

' Recognizing the pronunciation of hyperlinlced ^ 



terms by means of a 
Speech Recognition System 



I 



Determining the universal-time 
corresponding to the recognition of the term 



1803 



i 



Creating a record including 
-the universal time; 
- the name of hyperlinked term; 
' the address associated with hyperlinked term 



1804 



I 



Storing the record on a Speech Hyperiink-Time 
Table on the Speech Server 



1805 



Fig. 18: Method for creating a Speech Hyperlini(-Time Table 
on a Speech Server and for recognizing the pronunciation 
of hyperlinl(ed terms during a speech 
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^ Creating a Selections Hyperlink-Time Table^ 
for the speech on the auditor device 

\ — ' 
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Recording on the Selections Hyperlink-Time 
Table the address of the Speech Hyperlink-Time 
Table on a Speech Server 
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Listening to the speech given by the 
. speaker 



I 
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Perceiving a topic of interest for which 
additional information or service Is desired 
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Selecting a topic by entering a selection 
command on the auditor device 
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Determining the current unh^ersal time by means of 
an universal time device (for instance a GPS receiver) 
integrated or connected to the auditor device 
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Recording the universal time In the Selections 
I4yperlink-Time Table 
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Fig. 19: Method for creating a Selections Hyperlinlc-Time 
table and for selecting topics of interest in a speecli 
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Recessing the Speech Server from the auditor device^ 



2001 
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Sending to the Speech Server 
the recorded universal time 



-2002 



r ' 

- I ^ 2003 

Searching In the Speech Hyperlink-Time Tabled 

on the Speech Server for the hyperlinked 

term active at said recorded universal time 

t ~ — — 



Retrieving from the Speech Hyperlink-Time 
Table in the Speech Server the name and the 
address of the selected hyperfinked term 



2004 



T 



storing the retrieved hyperlink name and 
destination address (URL) into the corresponding 
entry of the Selections Hyperlink-Time Table 
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Selecting a hyperlink from the Selections 
Hyperlink-Time Table 



2006 
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Activating the hyperlink using a browser ^ 
program in the auditor device 

i • 
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Accessing the information and/or service 

I 
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2008 



Retrieving and displaying the information 
and/or service using the browser program 
on the auditor device 
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Fig. 20 : Method for accessing information related 
to selected terms 
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